Smart Surveillance

Abstract

Every individual in today\'s society has a need for a safe and reliable system. The most commonly used closed-circuit television (CCTV) or video surveillance systems are being implemented everywhere: hospitals, warehouses, parking lots, and buildings. However, this highly effective system has a cost disadvantage. Therefore, a cost-effective system is required. This project proposes to use a security camera with night vision using OpenCV. This is a cost-effective method. Images are captured and processed frame by frame. When a person is detected, the image is saved and an email is sent. The accuracy of this system is about 83%. It also uses the improved properties of the built-in camera. So the image is captured by the camera and sent to processing for face and person detection using OpenCV. Then, whether the detected person is known (visitor) or (stranger) () the detected face is compared against the database, and based on the output an email is generated and sent to the user. Security cameras can be controlled by custom AI. assistant. AI features include turning security cameras on and off with voice control. Accordingly, a low-cost security system can be provided.

Introduction

I. INTRODUCTION

Nowadays, people want one sole thing which is to make themselves feel safe and secure. The most commonly used security system is surveillance. The cost of implementation of surveillance varies depending upon the size and use of the system. It is usually installed in hospitals, malls, parking lots, etc.[1] However, with the help of surveillance one can monitor the area 24/7, or the footage if stored in a location can be retrieved when required. Although it can be used to deter crime and allows the authorities to identify and solve a crime, it doesn’t detect nor recognize the person who is involved.[2] We have implemented a system that provides both face detection and faces recognition with additional features with the help of AI and Computer Vision. Thus, when dealing with real-time image processing, Open source computer vision(OpenCV) software, a powerful library of image processing tools, is a good choice. With the help of a smart surveillance system, we have achieved a system that can record the event, and detect and recognize the person. [3]A database is used to take a record stating whether the person is an intruder or a visitor by capturing pictures during movement. If it is a visitor, then a picture is captured automatically by the camera to perform some operation like open the door (any type of automation is implemented) however if it is a stranger capture of an image is generated to indicate that there is an intruder. [4].

II. LITERATURE REVIEW

Literature survey is mainly carried out in order to analyze the background of the current project which helps to find out flaws in the existing system and guides on which unsolved problems we can work out. So, the following topics not only illustrate the background of the project but also uncover the problems and flaws which motivated us to propose solutions and work on this project.[5]

Arjunetal. present a survey of wireless sensor networks for Border Surveillance and Intruder Detection. The aim is to devise a multi-sensing system that is developed by combining different techniques of surveillance and intruder detection, for varying border scenarios such as flat surface movement or waterbody movement. Various sensors for detecting human intruders are discussed, such as geelyphones, hydrophones, infrared cameras, and surveillance cameras [6].

Palatieral. We propose a model that studies images captured by surveillance cameras, converts images into frames and extracts features. Key features are extracted using an ROI-based object tracking method. Finally, the extraction of semantic content makes the attacker aware without false matching.

Bhaskar introduces a framework that combines human detection, tracking, and human identification based on face recognition for surveillance purposes. Background subtraction is used to detect moving targets. Face recognition includes target face detections. If face detection fails, target tracking continues.

Jisaetal proposes an intruder detection system using the object detection method using a wireless sensor network. A passive infrared (PIR) sensor is used and is additionally connected to the MICAz sensor node. The proposed system is expected to detect and track intruders, as well as report speed and direction to the central base station for further processing.

Sagaretal. Implement the smart home robot security system by applying the image processing method. The system can recognize faces, sign, and notify the user if an intruder is detected. The Raspberry Pi is used to control the robot's movement via an Arduino, all sensors are connected wirelessly. [7]

Singh and Kushwaha propose a mechanism for intelligent border surveillance and automatic combat. It uses features extracted from scene optical flow information. : When an intruder is detected automatically, the appropriate action is taken based on the intruder's relative position to the perimeter fence. If the intruder is behind the fence, simple tracking is performed. An alarm is raised if an intruder is on the fence  and tries to cross it. Auto-fire can be activated when Intruder actually crosses a fence

Sun et al. Implement a hybrid wireless sensor network architecture for the border patrol system, called Border Sense. It consists of 3 layers. The first layer consists of underground and above-ground sensors. The second layer contains multimedia sensors that convey visual information. The third tier consists of drones that extend the range and flexibility. It also describes the distribution of system components.[8]

Zhang and Liang propose a new method to detect moving human bodies based on the technique of background subtraction. Initially, a background image is obtained. To extract the moving regions from the current frame,a difference between the current frame and the background image is obtained. At last, the shape features of the extracted regions are used to determine if the moving region is a human or not[9].

Ye et al. Present a method for detecting moving targets using background subtraction and shadow removal methods. This method applies to the RGB color space. The metric truncated mean and mean absolute deviation are estimates used to subtract the background value of. Saturation difference and brightness difference are shadow removal estimates of. [10]

Mishra et al. Deploy wireless sensor networks and artificial neural networks (ANNs) to develop and test systems for border intrusion detection. Distributed sensor motes send information to a central base station that contains an ANN for pattern detection () and self-learning using a backpropagation algorithm . Experimental results are presented for different cases with varying environmental conditions. [11]

Alcatamietal. Investigate the performance of border surveillance simulations using the WSN array. Provides experimental results for OPNET ( Optimized Network Tools) when using ZigBee nodes with wireless sensors. Grid and tree clusters are two topologies for conducting experiments and comparing results. Shivani and Kaur propose a way to detect border intruders using CCTV cameras. Scene ROI (region of interest) can be used to detect multiple events in surveillance video. A Brief Review Review of Existing Border Security Operations.[12]

III. PROBLEM IDENTIFICATION & CHALLENGES

Existing surveillance cameras have the following problems, so we improved in this topic. Issues such as: Occasionally older CCTV systems have simple wear issues. Direct sunlight on the outdoor camera can damage the camera over time. Also, older technologies that are currently not weather-resistant may experience hardware issues when exposed to urea or low/high temperatures. This applies to all technologies and there is very little you can do about it. Unfortunately, the isn't designed to last a lifetime unless it's an indoor camera, and the sometimes costs extra to protect your home and family.[14]

1. Night Vision Not Functioning Correctly- Sometimes the CCTV system works fantastic during the day and starts running into issues at night. In many cases the camera itself may have a light, but in any case the camera should be 100% illuminated at night time.
2. Using Non-Interchangeable Equipment and Building Your Own Bundle- Let's discuss one thing very clearly. I believe in completing 100% of my own systems and installing and assembling complete CCTV systems by myself. However, people often make the mistake of buying the cheapest DVR and pairing it with a nice doorbell camera and a bunch of cheap outdoor and indoor cameras. 33 Or buy a bunch of video surveillance equipment that's too cheap to be compatible in the first place. This will undoubtedly cause serious problems in doing whatever the system is designed to do.[15]
3. Not Taking the Time to Set up Security and Stronger Passwords- This is more of a security issue than anything else. Security systems, including the home video surveillance system, can be easily hacked. Especially if you are someone who sets their password as a "password" or uses very weak password combinations. : Start with stronger passwords and always perform regular software updates whenever possible. They are not released just for fun, they generally contain useful code and updates to provide what your system needs to run properly.[16]
4. Not Staying Up to Date with Hardware and Software- Many people bring their CCTV systems, install them, download mobile apps and never look back. The problem with this action is that these devices require update to work properly and sometimes you will need to purchase new hardware. This is how the world works with any technology or electronic device. A problem is occurring. is old and needs to be replaced. If you don't do this and do your best, either you are not truly protected with a or your system will be running at 100% capacity.
5. Protecting with More Than Just the Typical Camera Here and There- This goes back to having the right equipment to monitor the whole house, but another has to do with visibility and should cover as many areas as possible. . Covering not only the entry point, but possibly other areas of the facility, such as the driveway of the facility, the overall view of the backyard, and the view of the outbuildings such as a barn or garage. [17]

IV. PROBLEM SOLUTION

We are developing a front-end platform using ReactJS/ HTML and CSS through which we will integrate various module. Module such as :-

1. Monitoring
2. Noises
3. In/Out
4. Recording
5. Identify

Software Requirements are- Windows/Linux/Mac OS any version, hence it can run on any platform. Python3, it need python to be installed in your system to run this successfully. Packages in python -

a. openCV 

b. skimage 

c. numpy 

d. tkinter

A. Recording

In this module the normal recording will be done with noticing each and every aspect of the room. This function offers same functionality as the traditional CCTV offers. We used this so that owner will get the basic functions also with extra one. These modules will be created using OpenCV under the domain of Computer Vision and Artificial Intelligence. After that we’ll integrate on single front-end which we created using ReactJS/HTML and CSS.

B. In/Out

This is the feature which can detect if someone has entered in the room or gone out.

So it works using following steps: 1 – It first detect for noises in the frame. 2 – Then if any motion happen it find from which side does that happen either left or right. 3 – Last if checks if motion from left ended to right then its will detect it as entered and capture the frame. Or vise-versa.[27] So there is not complex mathematics going on around in this specific feature. So basically to know from which side does the motion happened we first detect for motion and later on we draw rectangle over noise and last step is we check the coordinates if those points lie on left side then it is classified as left motion.

C. Noises

This function is used to search for noise in the frame. This is what you'll find on the in most CCTV systems, but in this module we'll see how it works. Simply put, it continuously analyzes every frame and checks for noise. Noise tested at consecutive frames. In a nutshell, it analyzes the difference between the two images and the Contours (a motion boundary is detected) by making an absolute difference  between the two frames, there is no motion when there is no boundary and there is motion when there is a boundary. As you know, every image is an integer/floating point pixel value that reports a brightness of pixels, and likewise each pixel has this brightness value.[18]

Hardware Requirements are- In terms of hardware requirements there is not much required at all but still below requirements are must : Working PC or Laptop Webcam with drivers installed Flashlight/ LED if using this at night.

D. Monitoring

This feature is used to find the stolen one in the frame visible on the webcam. This means He constantly monitors the frame and sees if a thief has stolen anything or something from the frame. It uses structural similarity () to find differences between the two frames. When there is no noise (), the first two frames are captured, and when no more noise appears in the frame, the second frame is captured. SSIM is used as an indicator to measure the similarity between two given images. As this technique has been around since 2004, a lot of material exists explaining the theory behind SSIM but very few resources go deep into the details, that too specifically for a gradient-based implementation as SSIM is often used as a loss function. The Structural Similarity Index (SSIM) metric extracts 3 key features from an image: Luminance, Contrast, Structure [20]The comparison between the two images is performed on the basis of these 3 features.

The system calculates the structural similarity index () between two given images, which is a value between 1 and +1. A value of +1 indicates that the 2 given images are very similar or identical, and a value of 1 indicates that the 2 given images are very different. Often these values are scaled to the range [0, 1], where extrema have the same meaning. [22]

1. Luminance: Luminance is measured by averaging all pixel values. It is specified as μ(Mu) and has the formula:

2.  Structure: The structural comparison is done by using a consolidated formula (more on that later) but in essence, we divide the input signal with its standard deviation so that the result has unit standard deviation which allows for a more robust comparison.

Luckily, thanks to skimage package in python we dont have to replicate all this mathematical calculation in python since skimage has pre build feature that d o all of these tasks for us with just calling its inbuilt function. We just have to feed in two images/frames which we have captured earlier, so we just feed them in and its gives us out the masked image with the score.[23]

3. Identification: This feature is very useful for small projects. Used to determine whether people frames are known. This is done in two steps:

a. Find the face in the frame. 

b. Use the LBPH face recognizer algorithm to predict the person from the already trained model.

Conclusion

In a modern society where crime and anti-social behaviors are increasing, human security is paramount and cannot be solved at any cost. The security cameras can rescue us and it can make us feel a lot better to know that the person is watching us, and even if something goes wrong, we\'ll have solid evidence to support us. So, if you are planning to buy a camera for your home or work, the \"What kind of security camera should I buy?\"[24] So, the idea of smart surveillance can help everyone who fail to choose which camera they want for them. This camera have a functionality which mostly cameras don\'t have. This is having a club of more than one features and this also save cost of purchasing as people will take one camera rather than purchasing more than one camera.

References

[1] Wilson Feipeng Abaya, a Low cost smart security camera with night vision capability, De La Salle Univ., Manila, Philippines. [2] Andrew S. Tanenbaum:Operating Systems Design and Implementation, Prentice Hall, 2006 [3] About IPTV on Wikipedia http://en.wikipedia.org/wiki/IPTV [4] About VNC on Wikipedia http://en.wikipedia.org/wiki/Virtual_ Network_Computing [5] LibVNC server http://libvncserver.sourceforge.net [6] DirectFB documentation http://elinux.org/DirectFB [7] jointSPACE documentation http://sourceforge.net/apps/mediawiki/ jointspace/index.php?title=Main_Page [8] PuTTy on Wikipedia http://en.wikipedia.org/wiki/PuTTy [9] Nicola L. C Talbot and Gavin C. Cawley. A fast index assignment algorithm for robust vector quantisation of image data. In Proceedings of the I.E.E.E. International Conference on Image Processing, Santa Barbara, California, USA, October 1997. [10] Arjun, D., Indukala, P. K. & Menon, K. A. U. \"Border surveillance and intruder detection using wireless sensor networks: A brief survey,\" International Conference on Communication and Signal Processing (ICCSP), pp. 1125-1130, Chennai, 2017. 56 [11] Harish, Palagati, Subhashini, R. & Priya, K. \"Intruder detection by extracting semantic content from surveillance videos,\" In Green Computing Communication and Electrical Engineering (ICGCCEE), 2014 International Conference on, pp. 1-5. IEEE, 2014. [12] Bhaskar, Harish. \"Integrated human target detection, identification and tracking for surveillance applications,\" 6th IEEE International Conference Intelligent Systems, pp. 467-475, Sofia, 2012. [13] Jisha, R. C., Ramesh, Maneesha V. & Lekshmi, G. S. \"Intruder tracking using wireless sensor network,\" In Computational Intelligence and Computing Research (ICCIC), 2010 IEEE International Conference on, pp. 1-5. IEEE, 2010. [14] Sagar, R N, Sharmila, S P, Suma, B V. \"Smart Home Intruder Detection System,\" International Journal of Advanced Research in Computer Engineering & Technology (IJARCET), 6(4), pp. 2278 – 1323, 2017. [15] Singh, Dushyant & Kushwaha, Dharmender. \"Automatic Intruder Combat System: A way to Smart Border Surveillance\", Defence Science Journal, 67(1), pp. 50, 2016. [16] Sun, Zhi, Wang, Pu, Vuran, Mehmet C., Al-Rodhaan, Mznah A., AlDhelaan, Abdullah M. & Akyildiz, Ian F. \"BorderSense: Border patrol through advanced wireless sensor networks,\" Ad Hoc Networks 9, 3, pp. 468-477, 2011. [17] Jin, Xin, Sarkar, Soumalya, Ray, Asok, Gupta, Shalabh & Damarla, Thyagaraju. \"Target detection and classification using seismic and PIR sensors,\" IEEE Sensors Journal 12, 6, pp. 1709-1718, IEEE, 2012. [18] Zhang, Lijing & Liang, Yingli. \"Motion human detection based on background subtraction,\" 2010 Second International Workshop on Education Technology and Computer Science, pp. 284-287, 2010. 57 [19] Ye, Jianhua, Tao Gao, and Jun, Zhang. \"Moving object detection with background subtraction and shadow removal,\" 2012 9th International Conference on Fuzzy Systems and Knowledge Discovery, pp. 1859- 1863, 2012. [20] Mishra A., Sudan K. and Soliman H., \"Detecting border intrusion using wireless sensor network and artificial neural network,\" 2010 6th IEEE International Conference on Distributed Computing in Sensor Systems Workshops (DCOSSW), pp. 1-6. IEEE, 2010. [21] Alazzawi, Lubna, Alkhathami, Mosad and Elkateeb, Ali. \"Border Surveillance and Intrusion Detection Using Wireless Sensor Networks,\" International Journal of Advances in Engineering & Technology, 8, pp. 17-29, 2015. [22] Shivani & Kaur, Lakhwinder. \"A Brief Review: Border Intruder Detection System Based On Computer Vision,\" Imperial Journal of Interdisciplinary Research, 2(7), pp. 2454-1362, 2016. [23] Felemban, E. \"Advanced Border Intrusion Detection and Surveillance Using Wireless Sensor Network Technology,\" International Journal of Communications, Network and System Sciences, 6(5), pp. 251-259, 2013. [24] Essendorfer, Barbara, Monari, Eduardo & Wanning, Heiko. (2009). \"An Integrated System for Border Surveillance,\" in Fourth International Conference on Systems, pp. 96, 2009.

Copyright

Copyright © 2022 Sandeep Singh, Priyansh Jain, Sneha Kesarwani, Vanshi Tiwari. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.